PCA

The goal of the project is to use a dimension reduction approach (PCA, etc.) to decompose and transform high-dimension image data into a (much) lower dimension space for classification and clustering. The dataset can be found here: https://drive.google.com/drive/folders/1FcSHxEKHIyZ_Vckh6K1GDTN-VdM7tgD6

For each subtask listed below, find the best two eigenvectors that define a 2D space to best separate corresponding categories, visualize individual data points in a 2D plot and calculate a Silhouette score for each subtask.

Subtask 1. Emotion recognition
Subtask 2. Mask detection
Subtask 3. Person identification

Bonus points: Instead of using a whole image, you are encouraged to try other (creative) ways to solve each problem. A better solution should have a higher Silhouette score than the default one. Even if you may not be able to implement an idea you come up with, you will earn extra points by clearly describing a new idea and explaining in what ways it may improve performance.

Write a report to explain your results and summarize what you’ve learned by trying to find the best dimension reduction solution for each problem.

Contents

Method 1: Grayscale image
Method 2: Edges
Method 3: Crop Image

Load data

Method 1: Grayscale Image

Subtask 1. Emotion recognition

Subtask 2. Mask recognition

Subtask 3. Person recognition

Method 2: Edges

Subtask 1. Emotion recognition

Subtask 2. Mask recognition

Subtask 3. Person recognition

Method 3: Crop

Subtask 1. Emotion recognition

Subtask 2. Mask recognition

Subtask 3. Person recognition

Report

For this assignment, I used PCA to decompose 112 face images into 10 eigenfaces. I tried finding the optimal number of components to use to reach 80% accuracy, but decided instead that I wanted to keep the number of components the same to be able to compare each of the image preprocessing methods easier. I used the silhouette score to evaluate PCA performance after three preprocessing alternatives:

1) raw grayscale images
2) image edges
3) cropped images (focusing on eye to lip area)

For the emotion subtask, the silhouette score was lowest when PCA was applied on the grayscale images. Cropped images yielded the second best performance, with edge images yielding the worst performance. This suggests that the algorithm was best able to make use of the full grayscale image in order to provide information about the emotion. It could be that a different edge threshold would have produced better results. Several edge thresholds were piloted, and I selected the one that I thought conveyed useful information but minimized extraneous information. It could be that the algorithm has a different idea about what constitutes useful information.

For the mask classification, the edge images yielded the best PCA performance, followed by the cropped images, followed by the raw. Because the mask is a fairly large, undetailed object, it could be that using the edges maximized the amount of information available about the mask presence over and above what the raw or cropped images provided. It was interesting to me that the cropped images performed better than the raw images here, since the majority of the mask area was removed. The remaining pixels containing mask vs. no mask data remained sufficient, and even relatively more useful, for the classification task.

Similar results were obtained for person classification as were found for the mask classification. Again, PCA performed on the edge images yielded the best results, followed by the cropped images, followed by the raw images.

In addition to using the silhouette score, I also fed the PCA results to a SVM classifier and used that to give predictive labels to images in the training dataset.

References

Sandipanweb Eigenfaces Tutorial